Method for reconfiguring a router

ABSTRACT

In a system which includes a configurable router which is configurable both as an edge router and a hub/backbone router and also including a router special needs agent (RSNA), a method which includes performing the steps of turning on active CoS/QoS provisioning between the configurable router and the RSNA such that the RSNA provides active cost of service (CoS)/quality of service (QoS) provisioning for packets entering the configurable router and such that the configurable router operates in an edge router mode and turning off (CoS)/quality of service (QoS) provisioning by the RSNA such that the configurable router operates in a hub/backbone router mode.

RELATED APPLICATIONS

Not applicable.

GOVERNMENT RIGHTS

Not Applicable.

FIELD OF THE INVENTION

This invention relates generally to routers and more particularly to ascalable and bi-directionally reconfigurable router architecture.

BACKGROUND OF THE INVENTION

As is known in the art, “a computer network” or more simply “a network”is a connection of computer systems and peripheral devices that exchangedata (or more generally information) as necessary to perform thespecific function of the network. To exchange information, networksoperate in accordance with a set of semantic and syntactic rulesreferred to as a protocol. One particular type of protocol is theTransmission Control Protocol/Internet Protocol (TCP/IP). When a networkoperates in accordance with the TCP/IP protocol it is typically referredto as an IP network.

As is also known, a router is a device which interconnects two computernetworks. When the networks correspond to IP networks, the routerinterconnects two computer networks that use a single network layer(layer 3) procedure but may use different data link layer and physicallayer procedures. Routers are protocol dependent because they must beable to identify an address field within a specific network layerprotocol.

As improvements in network protocols, hardware speed and features occur,it is often necessary to upgrade routers to take advantage of suchimprovements. Thus, router hardware (HW) upgrade or more generally,network hardware expansion to accommodate end user demand on higherspeed/capacity or more advanced set of QoS features is an importantprocess to an IP network as well as other types of networks. Withrespect to IP networks, network hardware expansions/upgrades must bedone to maintain an internet service provider's (ISP's) ability tocontinue to offer state-of-the-art value-added (and sometimes bandwidthintensive) IP services.

There are in general three types of layers in an ISP network hierarchy:“edge,” “hub,” and “backbone.” Since the functionality required forevery layer is different, to upgrade or expand router hardware, it mustbe done separately for each individual network layer. That is, routerfunctionality required of an “edge” layer is different from routerfunctionality required of a “hub” layer, which in turn is different fromthe router functionality required of a “backbone” layer. Hence internalarchitectures amongst the edge, hub or backbone routers today are vastlydifferent from one another. With this difference in functionality andinternal architecture, to perform field upgrades, it is often necessaryto acquire new routers for each individual layer in the networkhierarchy. The new routers also often need to be lab-tested to ensureinteroperability with existing hardware and software, an oftentime-consuming effort before field deployment can be made possible.

Assume, for example, that it is desired to perform network hardwareexpansion to add capacity to a router to accommodate more end users.This translates to adding more input/output (I/O) cards to expand therouter's port capacity. However, it is not possible to add more I/Ocards “on demand” without a new switch fabric (SF) replacement if thecapacity for target expansion were not already built into the router SFat its first implementation. This is because capacity of mostconventional router SFs (e.g. bus or crossbar) cannot be easily expandedon demand. Hence, considering the time and manpower consumption requiredto implement a replacement router SF, it is often less expensive toacquire a new router rather than utilizing the router SF replacement.

Moreover, the functionalities of “edge”, “hub” and “backbone” routerscannot be interchanged without also undertaking the time-consuming taskof altering internal router hardware architectures required to adapt tothe functionality of each router at the “target” network layer.Consequently, other routers in the network are typically not availableas hardware resources to rapidly address upgrade or expansion needs ofthe different layers.

For example, an “edge” router cannot become a “hub” or “backbone” routerwithout first undertaking a number of time-consuming andmanpower-intensive modifications such as reconfiguring internal hardwarearchitecture to take off or disable all internal quality of service(QoS) control plane functionalities and associated memories and database(for example, those for traffic policies/rules and network resourceadministration and management). Depending upon the original routerhardware architectural implementation, this may involve physicalalteration of internal hardware components such as dismantling ofdatabases (e.g., tables) mounted on a computer board and installing newlookup functions to adapt to the hardware architectures appropriate foruse as a backbone, and modification of internal packet-processing datapaths through the router. In particular, it may be required to disableor dismantle physical wires that lead to a policy database/lookup tablethat is no longer useful in a “backbone” environment (e.g a policydatabase/lookup table such as packet prioritization based on rules).Also, if the data path were originally pipelined, the pipeline wouldneed redesign since some pipeline stages must be skipped (or taken out)due to the disabling of the above-mentioned hardware components.Furthermore, such a pipeline modification may impact and thereby inducea change in the router's clock speed.

Furthermore, each of the above steps is labor-intensive and thus costlyand time-consuming. Hence, upgrade of backbone routers using edgerouters is rarely if ever done due to the labor and expense of hardwarearchitectural alteration/re-work necessary stemming from the completelydifferent internal hardware designs for “edge” and “hub/backbone”routers in most vendor products. Thus, when a “backbone” routerupgrade/expansion is desired or necessary, a new router is typicallypurchased.

Similarly, it is also labor-intensive to adapt a “backbone” or “hub”router today for use as an “edge” router as it would entail the reversalof a number of the tasks listed above. For example, the process wouldinclude reconfiguring the router internal hardware architecture byinstalling quality of service (QoS) control plane functionalities andassociated memories/databases on policies/rules, and network resourceadministration and management. Also, depending upon the desired targetrouter hardware architectural implementation, the above-mentionedreconfiguration may also involve installing added databases and rewiringto accommodate added physical wires leading to the addeddatabases/memories on a computer board, and if the packet-processingdata path was pipelined, the pipeline must be redesigned to accommodateadditional stages of database fetching/reading, making the pipelinelonger and thus more complex, which could result in degradation of theoverall performance of a router.

One attempt to improve network performance has been to reduce the numberof layers. The Cisco 12000 Series IP Server Engine, available from CiscoSystems, Inc. of San Jose Calif., leverages the distributed architectureof the chassis with features that allow service providers to reduce boththe number of layers and the number of boxes in the POP. It collapsesthe access/aggregation and distribution layers into a single layer withdense (channelized 2.5 Gbps), all-optical aggregation of existingtransmission technologies on the customer side and 10 Gbps upstreamconnections within the POP. The network is less complex because thereare fewer overall connections and fewer devices to manage.

Another problem which occurs is that a capacity/speed conversion orupgrade of an ISP network having multiple layers of network hierarchyoften involves putting together different vendor equipment for thedifferent network layers, as one vendor may not supply routers withdifferent functionalities required for use in different layers of thenetwork. For example, one vendor may specialize only in supplyingrouters for use at the “edge” layer but not the “backbone” layer, andvice versa. In such situations, lengthy and expensive networkinteroperability testing and field (confidence) testing must beconducted for the different equipment from different vendors in thedifferent network layers. This results in a relatively long lead-timebeing required in the hardware expansion/upgrade process for all networklayers. This is especially true when the expansion/upgrade involves newnetwork features/functionalities. Hence, since router hardwareexpansion/upgrades must often be done network-wide for all networklayers to ensure interoperability after the expansion/upgrade, theproblem of costly and manpower intensive router hardware upgrades isexasperated with an increase in the numbers of network layers.

Another problem is that since IP QoS control is distributed in eachindividual router in conventional systems, only hop-by-hop non-optimal,instead of global, view of QoS control is possible. With QoS control inits distributed form, conventional router hardware architectures cannotscale to include whole-path data storage capacity needed for networkwhole-path QoS provisioning within the router itself due to router sizeand/or central office space constraints, among other inefficiencies. Thereason being that to provision conventional routers with globalknowledge for network whole-path-related QoS control would mean enormousexpense since it would be necessary to equip each router with numerousbuilt-in databases that yield such network-wide information. Examplesare: policy information base, path QoS state information base, flowinformation base, multicast tree/state control server, multicasttree/state information base, policy control server, among others. Suchprovisioning is difficult to achieve within the limited space of arouter chassis and with equipment footprint size restriction inside thelimited space of a central office for routers.

Moreover, frequent and time-consuming synchronization of states amongstrouters (often by message exchange in the form of a network protocol) isneeded to ensure network-wide data/state coherence. With the inabilityto scale efficiently to include whole-path data storage capacity neededfor network-wide QoS provisioning, an individual router has no globalview of the bandwidth resource for arbitrary routes in the network.Consequently, it is highly inefficient for a router to provisionvalue-added overall network-wide whole-path-related services such asIP-based VPN which involve routes elsewhere in the network the bandwidthresource status of which is not known to the router unless the routerspends the time to “ask” for it via message exchange in the form ofspecific protocols.

In view of the above, it is clear that conventional router hardwarearchitectures are not designed for cost- and manpower-efficient networkhardware expansions or upgrades. In most cases in which a networkhardware expansion/upgrade corresponds to a router hardwareexpansion/upgrade, the router hardware expansion/upgrade is synonymouswith purchasing new routers. This approach results in the expenditure oflarge amounts of capital and manpower resources, as well as long hoursof “costly” downtime for all parties involved (e.g. ISPs and end users).

The conventional processes for performing a network hardware expansionor upgrade are cost-inefficient and manpower intensive. Consequently,many companies (e.g. AT&T, Corp.) spend a tremendous amount of energyand resources in the process of network hardware expansions/upgrades.

It would, therefore, be desirable to provide a system which allowsnetwork hardware expansion or upgrades (including router hardwareexpansions or upgrades) to be accomplished efficiently in terms ofdollar and manpower resources. It would also be desirable to provide asystem and technique which allows a router to be reconfigured as an“edge”, a “hub” or a “backbone” router on-demand in order to induceefficient network hardware expansion or upgrades even in the face ofnetwork layering complexity (e.g. ISP network layering complexity).

SUMMARY OF THE INVENTION

In accordance with the present invention, a multi-path router includes afilter for receiving a packet header (either the header alone or as partof the whole packet) and determining the needs of the packet header andfor selecting one of: (1) a time-sensitive path, (2) anon-time-sensitive path and (3) a special needs path in response to theneeds of the packet header. The multi-path router also includes a RouterSpecial Needs Agent (RSNA), coupled to the filter and adapted to receivethe packet headers (either alone or as part of the whole packets) havingspecial needs services.

With this particular arrangement, a multi-path router capable of rapidbi-directional hardware reconfigurability is provided. The RSNAcomponent primarily performs the “edge” layer/(Quality of Service) QoScontrol plane functions and administers management and control forheaders of packets with special (e.g., priority/network resource)requirements. In addition to QoS control plane, the RSNA also performssome data plane functions in the processing of the packet headers. Byutilizing the RSNA as part of an overall router hardware architecture,the router is quickly “bi-directionally” interchangeable either downwardas “edge router”, or upward as a “hub or “backbone router,” and providesan immediate hardware resource suitable for use in all layers of anetwork hierarchy such as an internet service provider (ISP) networkhierarchy, for example. The multi-path router thus allows speedy and yetmanpower and cost-efficient network hardware expansions/upgrades andnetwork emergency fixes/responses to be performed which is not possiblewith conventional router hardware architectures.

The router hardware architectures of the present invention operate asfollows. Every router in the network has a connection to the RSNA (whichcould be either internal or external to the router) and every router hastwo different operating modes: (1) an “edge mode” and (2) a“hub/backbone mode.” These two modes are mutually exclusive of eachother. A router becomes an “edge” router and serves the edge functionsby activating the “edge” mode within the router. This corresponds toturning on active CoS/QoS provisioning between the configurable routerand the RSNA such that the RSNA provides active CoS/QoS provisioning forpackets entering the router and such that the router operates in an edgerouter mode. Likewise, a router becomes a “hub” or “backbone” router byturning on the “hub/backbone” mode within the router. This correspondsto turning off active CoS/QoS provisioning by the RSNA such that therouter operates in a hub/backbone router mode. This mode can includepassive CoS/QoS provisioning. An I/O card speed adjustment for the twodifferent modes can be done via reprogramming of a re-programmableinterface module (IM) receiver (i.e., the I/O card “slot”) through acraft graphical user interface (GUI).

The multi-path router also includes an interface module (IM) (or linecard) adapted to be disposed in the IM receiver. The IM includes aforwarding engine. The packet/header processing functions within theforwarding engine located in each IM (e.g. header filtering, tablelookup, packet queuing and scheduling) are provided from re-programmablehardware-based chips (e.g. field programmable gate arrays (FPGAs) and/orapplication specific integrated circuits (ASICs)) arranged in pipelineformat to achieve very high speed processing; for example, one task perCPU cycle is possible, independent of incoming traffic rate.

Internal packet header processing functions of the RSNA are also carriedout via re-programmable hardware-based components (e.g., FPGAs, ASICs)arranged in pipeline format for very high speed packet (header)processing. Exemplary functions carried out via the programmablehardware-based components are packet header filtering, packet dropqueuing, and packet drop priority table lookup. Thus, includingreprogrammable components in the routers enables router adaptationon-demand to the constant changes of network features including IPnetwork features.

The router also includes a network interconnect module (NIM), i.e., aswitch fabric. Three candidate switch fabrics can be used in the routerarchitectures described hereinbelow. One switch fabric is of the“indirect” interconnection network (IN) family—the Bi-directionalMultistage Interconnection Network (BMIN). Two other switch fabrics areof the “direct” IN k-ary n-cube family—the 2-ary, n-cube or hypercube,and k-ary 3-cube or 3-D Torus. All three switch fabric candidates aremodular and can be expanded on-demand. However, expansion of the k-aryn-cubes requires that the desired IM/processor interface capability andchannel bandwidth at the maximum expanded capacity to be pre-allocatedto existing IMs and channels at the first implementation. The underlyingreason is to accommodate the increase in traffic volume that comes withan increase in the number of IM resources after the expansion, whereasexpansion of the BMIN can be done without disturbing the existingphysical capacity of the existing MIN infrastructure.

“Edge” mode, “hub/backbone” mode, and their relationship to the RSNA arefurther explained below. Briefly, however, the “hub/backbone” modeenables the router to filter incoming packets based on packet header“tags” that are previously administered by the RSNA in communicationwith the upstream “edge” router from which the packet came. In addition,depending upon implementation, this mode may either enable the router'sRSNA connection to be turned “off” altogether or to still be “on” andenter passive QoS/CoS provisioning for packets. An example ofimplementation for this latter case could be as follows. When a routerturns on the “hub/backbone” mode, it automatically sends a message toinform the RSNA that it is now in “hub/backbone” mode, and startslistening to and executing commands from the RSNA. The RSNA may request,for example, that the router reserve certain bandwidth capacity for anupcoming stream of packets with certain header “tags.”

When the router operates in the “edge” mode, this enables the router'sRSNA connection to be “on” which allows the router to communicate withthe RSNA via a request/response session. A request to the RSNA is apacket header that needs classification or priority information. Aresponse from the RSNA is a newly updated/“tagged” packet header.Moreover, “edge” mode operation enables the router itself to filterheaders of incoming packets based on a predetermined set of rules, e.g.,some combination of the following 5-tuples: source/destination IPaddresses, source/destination port numbers and protocol, to determine ifit's one of: (1) a “special needs” packet, (2) a “non-time-sensitive”packet; and (3) a “regular” packet.

The details of each of the three types of packets mentioned above aswell as the processing of each will be described below in conjunctionwith the figures. In general overview, however, the packet headers areseparated into different logical paths for processing (e.g. one logicalpath for “special needs” packets, one logical path for“non-time-sensitive” packets, and one logical path, that is, the“time-sensitive” path, for “regular” packets). As will be explainedbelow, in some cases the headers are updated while on the respectivedifferent logical processing paths. Once the headers return from theirrespective path, they are rejoined to their respective payloads and thewhole packets go through the “time-sensitive” logical processing path ofthe router which physically comprises the router's data plane. In otherwords, all packets eventually go through the “time sensitive” logicalprocessing path of the router. One exception to this rule is for thosepackets destined for the Network Processor Module (NPM) of the router.The overall effect of this path separation for processing packet headerswith different needs is, hardware-wise, a much simplified and speedydata plane for processing all packets passing through the router.

The above approach using the ON/OFF capability to the RSNA provides arouter with rapid bi-directional reconfigurability which can be utilizedeither during either a network hardware expansion/upgrade or emergencysituations, by turning on the router's “edge” mode or “hub/backbone”mode according to need (e.g. according to the need of an ISP).

During emergency situations, for “edge” routers to act as a temporary“hub” or “backbone” router, the process includes grouping togetherseveral “edge” routers to meet the throughput requirements of a “hub” or“backbone” router, turning on each router's “hub/backbone” mode andadjusting each router's I/O line rates via reprogramming of the IMreceivers as needed.

During emergency situations for “hub/backbone” routers to act astemporary “edge” routers, the process includes turning on the “edge”mode of the hub/backbone” router, and adjusting the router's I/O linerates via reprogramming of the IM receivers as needed to provide adesired (ideally the quickest) emergency fix/response turn-around time.

One example of a network emergency situation occurs if some “hub” or“backbone” routers unexpectedly stop operating properly or at all. Inthis case, the network hierarchy can be quickly reconfigured such thatsome “nearby” “edge” routers can temporarily take over as “hub” or“backbone” routers and continue functioning, and vice-versa for theunexpected stop of an “edge.”

During a network hardware expansion/upgrade operation to increasetraffic throughput for “hub/backbone” routers, the process toautomatically become an “edge” router includes turning on the“hub/backbone” router's “edge” mode and adjusting via reprogramming theI/O line rates of the IM receivers as needed to provide a desired(ideally the quickest) provisioning turn-around time.

During a network hardware expansion/upgrade operation to increasetraffic throughput for “edge” routers to become “hub/backbone” routers,the process includes turning on the “hub/backbone” mode of the routerand adjusting line rates of the IM receivers as needed and eithergrouping together several “edge” routers to meet the throughputrequirements of a “hub” or “backbone” router, or if time permits, byadding more IM receivers and wiring connections to expand the routerswitch fabric. It should be noted that the latter is possible only ifthe existing router's CPU has adequate processing power to handle thetarget line rates.

By providing a router which can be rapidly bi-directionally reconfiguredduring network hardware expansion/upgrade or emergency situations (i.e.where a “hub” or “backbone router” can quickly turn into an “edge”router and vice-versa) the router architecture and processes of thepresent invention make network hardware expansion/upgrade and emergencyfix/response independent of the number of layers within the networkhierarchy (e.g. an ISP network hierarchy). This characteristic isdifficult, if not impossible, to achieve with conventional routerhardware architectures. In other words, the systems and processesdescribed herein enable rapid, efficient network hardwareexpansion/upgrades and network emergency fix/responses even in the faceof network layering complexity (e.g. ISP network layering complexity).The present invention thus not only saves manpower and capital-costs,also reduces the network downtime intervals during these network eventsfor all parties involved (e.g. ISPs and end users).

Furthermore, if the RSNA is implemented as an independent networkcomponent physically separate from the router, the invention yields theadded benefit of a flexible, scalable and cost-effective hardwarearchitecture which provides routers with global knowledge andinformation on an overall network path view needed to provision servicesthat require network-wide QoS control. The router architecturesdescribed herein thus also enables routers to cost-effectively provisioninternet protocol (IP) services with quality of service (QoS) controlsthat require overall global network path view, such as customer-specificvirtual private networks (VPNs), with individually differentservice-level agreements. Network whole-path based QoS control isdifficult to achieve with conventional router hardware architecturessince the QoS control is in distributed form within each individualrouter. Examples of overall global information are network-wide pathresource reservation, capacity and path-state, these are needed tosupport optimal, meaningful network whole-path-based QoS serviceprovisioning, such as multicasting and VPN.

The rapid bi-directional reconfigurability/interchangeability fornetwork hardware expansion/upgrade and emergencies, and other benefitsmentioned above, cannot be achieved with conventional router hardwarearchitectures. This is especially the case with network emergencies,such as sudden router malfunctioning or death since a common currentsolution of such a situation relies on network protocols (e.g., OSPF) toinform all other network routers of the topology update, to avoid thetroubled router(s).

The present invention also results in lower cost to perform networkhardware expansion/upgrades including router hardwareexpansion/upgrades. In particular, significant savings in capital costis achievable with the novel router architectures described herein sinceinstead of having to buy new routers for network hardwareexpansion/upgrade and providing spares for network emergencies as iscurrently done, existing network routers can be reused and expandedupon. Further, additional savings are provided since the above describedarchitecture using the RSNA significantly reduces cost (e.g., testing),manpower and turn-around time for network hardware expansion/upgrade andnetwork emergency situations of every layer within the networkhierarchy. Thus, having a router hardware architecture which can berapidly scaled and bi-directionally reconfigured for all layers of thenetwork hierarchy (i.e. edge, hub or backbone) simplifies, streamlinesand reduces costs associated with expansion/upgrade and emergencyfix/response process.

In one embodiment, a multi-path router hardware architecture referred toas Baseline Reconfigurable Limited Scalable (BRLS) router which adheresto the concept of hardware scalability and bi-directional hardwarereconfigurability via use of an internal RSNA is described.

The BRLS model serves as a baseline architecture from which otherhardware architectures for a scalable bi-directionally reconfigurablerouter can be developed. Such an architecture may be used to provide,for example, a scalable bi-directionally reconfigurable InternetProtocol (IP) router. An IP router provided in accordance with the BRLSmodel can be applied as a matching router hardware solution for thenewly emerging concept of a single-layer IP network architecture.

The BRLS also utilizes a switch fabric that is capable of incrementalexpandability. The BRLS uses switch fabric hardware, and re-programmablehardware capabilities that allow adaptation “on-demand” to the changingneeds of IP and other networks.

It should be appreciated, however, that architectural scalability of aBRLS router is largely limited to the router's data plane only and notthe QoS control plane. This is due, at least in part, to the fact thatit has a non-distributed router architecture with RSNA as part of theinternal components of the router. The consequence is that the QoScontrol plane can't scale to include network whole path data storagecapacity needed for network whole path QoS provisioning, but onlyhop-by-hop view for QoS control is possible.

In another embodiment, a multi-path router architecture which utilizesan external RSNA (i.e. an RSNA which is external to the router hardwareitself) is referred to herein as a one-architecture-fit-all scalablereconfigurable router (OSR). The RSNA is a distinct server/databaseentity responsible for the CoS/QoS control plane that dynamicallyprovides packets with resources and reservation information pertainingto an overall network view.

Like the BRLS architecture, the OSR architecture may also be used toprovide, for example, a scalable bi-directionally reconfigurableInternet Protocol (IP) router which provides a router hardware solutionfor the newly emerging concept of a single-layer IP networkarchitecture.

The OSR achieves bi-directional hardware reconfigurability by its use ofthe external RSNA and also utilizes a switch fabric that is capable ofincremental expandability, and thus corresponds to a scalable,bi-directionally reconfigurable router. The OSR uses switch fabrichardware, and re-programmable hardware capabilities that allowadaptation “on-demand” to the changing needs of IP and other networks.

It should be appreciated, however, that architectural scalability of anOSR router is not limited to the router's data plane and can extend tothe QoS control plane. This is due, at least in part, to the fact thatthe RSNA is a physically separate entity that can serve more than oneOSR router in its network domain of responsibility and has overall(network) view of network topology information. The consequence is thatthe QoS control plane can scale to include network whole path datastorage capacity needed for network whole path QoS provisioning.

For both the BRLS and OSR embodiments, the RSNA function is always onfor a BRLS or an OSR router designated as “edge”, but may be off for aBRLS or an OSR router designated as either “hub” or “backbone.”Bi-directional reconfigurability immediately follows with this on/offflexibility to the RSNA, which can be used either during networkhardware expansion/upgrade or emergency situations.

The BRLS and OSR hardware architectures make use of currently availablestate-of-the-art technology in the market place. The BRLS and OSRarchitectures are each arranged in a specific way to achieve speedybi-directional reconfigurability/interchangeability to perform differentroles on-demand depending on the router's desired physical placement inthe network hierarchy. The architectures also provide for efficientscalability to adjust to network capacity growth demand with flexibleadaptation to changing network features. End users benefit fromshortened network hardware expansion/upgrade downtime interval. ISPs andother service providers benefit from shortened network hardwareexpansion/upgrade downtime interval and ability to harness routertechnology growth as well as other network technology growth. Networkhardware and router vendors/developers benefit from the ability todirectly address service provider needs.

In still another embodiment, an architecture utilizes a hybridreprogrammable chip technology with one being electronic FPGAs or ASICsand the other optoelectronic FPGAs. The architecture also uses a sensortechnology to sense incoming traffic speed and based on the sensorresult, to route traffic accordingly either via the “almost all-optical”or the “electrical/optical” hardware technology path through the router.The only non-optical part in the “almost-all-optical” processing path isthe optoelectronic devices that implement the router's data planefunctions within the IMs. This architecture provides all of the benefitsof the BRLS and OSR architectures as explained above, as well as thebenefit of an “almost all-optical” incoming-traffic-speed-independenthardware path for packet processing through the router.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of the invention, as well as the invention itselfmay be more fully understood from the following detailed description ofthe drawings, in which:

FIG. 1 is a block diagram of a network having a reconfigurable router;

FIG. 2 is a flow diagram for internet protocol (IP) packet forwardingtasks analysis;

FIG. 3 is a block diagram of an architecture for a BaselineReconfigurable Limited Scalable (BRLS) Router;

FIG. 4 is a functional block diagram of a Baseline ReconfigurableLimited Scalable (BRLS) Router interface module (IM) and router whichuses K-ary N-cube as the switch fabric;

FIG. 5 is a functional block diagram of a BRLS Router Internal showingthree logical packet/header processing paths;

FIG. 6 is an overall network view diagram of a One-Architecture-Fit-AllScalable Reconfigurable Router (OSR);

FIG. 7 is a block diagram of an architecture for aOne-Architecture-Fit-All Scalable Reconfigurable (OSR) router;

FIG. 8 is a functional block diagram of an OSR router using K-ary N-cubeas the switch fabric and showing three logical packet/header processingpaths;

FIG. 9 is a functional block diagram of a Router Special Needs Agent(RSNA) having an OSR or optoelectronic one-architecture-fit-all scalablereconfigurable (OOSR) router architecture;

FIG. 10 is a functional block diagram of an optoelectronicone-architecture-fit-all scalable reconfigurable (OOSR) router showing aspeed-sensing general-purpose interface module (IM) receiver and dualintra-IM interconnection technologies; and

FIG. 11 is a functional block diagram of an OptoelectronicOne-architecture-fit-all Scalable Reconfigurable (OOSR) Router.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Before describing the present invention, it should be understood thatreference is sometimes made herein to operation of the present inventionin connection with internet protocol (IP) networks. Such reference isintended only for clarity and clearness in explanation and descriptionand should not be construed as limiting. It should be understood bythose of ordinary skill in the art, that the concepts, apparatus andtechniques described herein find application in a wide variety ofnetworks including but not limited to IP networks. Reference is alsosometimes made herein to forwarding a packet header to a particularlocation within a router or to a router special needs agent (RSNA) or toa certain physical or logical path. It should also be understood that insome applications it may be desirable or even necessary to forward theentire packet rather than the packet header alone. Similarly, whenreference is made to forwarding or processing a payload, it should beunderstood that in some applications it may be desirable or evennecessary to forward the entire packet rather than the payload itself.Also, in some instances certain components/elements referred to hereinbelow are described as being either implemented as hardware-basedfunctional units or implemented as processors (i.e. implemented viasoftware). It should be understood by those of ordinary skill in the artthat most components/elements can be implemented in either hardware orsoftware and that a variety of factors are considered in determiningwhether it is preferable to implement any particular components/elementsin hardware or software.

Referring now to FIG. 1, a first network 10 is coupled through amulti-path router 12 to a second network 14. The first network maycorrespond, for example, to a local area network (LAN) and the secondnetwork may correspond, for example, to an internet. As will beexplained in further detail below in conjunction with FIGS. 2–11, themulti-path router 12 is “configurable” meaning that it has the abilityto be configured (or re-configured) “downward” as “edge” or “upward” as“hub/backbone” router on demand. Additionally, after being configured asan “edge” or a “hub,” the router can be re-configured back to itsoriginal arrangement/function. This means that if the router wasoriginally arranged as an “edge” router it can be configured as a “hub”router and then later re-configured back to an “edge” router again. Thereverse is also true, that is, the router may first be configured as a“hub/backbone” and then “configured” (or “re-configured”) as an “edge”.The router can be repeatedly configured and re-configured between “edge”and “hub/backbone” router configurations. Thus, the router is also saidto be “bi-directionally configurable” (or “bi-directionallyre-configurable).” With these characteristics, the router can beversatile depending upon its physical placement within a networkhierarchy. The multi-path router of the present invention can beprovided using a methodology for reconfigurability and seven basicprinciples of hardware scalability and hardware reconfigurability asdescribed by E. S. H. Tse-Au in “Design and Analysis of ScalableReconfigurable IP Routers”, Ph.D. Dissertation, Dept. of ComputerScience, Stevens Institute of Technology, Nov. 19, 2001.

Referring now to FIG. 2, a flow diagram of the fundamental packetprocessing/forwarding functions in a router is shown. Theforwarding/processing operation of a packet 16 can be fundamentallypartitioned into eight steps which reflect the basic requirements for IPheader processing. As shown in steps 18 and 20, an incoming packet isreceived and processed for level two (L2) and level three (L3)functions.

As part of the L3 functions, the packet header is first filtered asshown in step 22. The filtering includes but is not limited to basicerror, header length and IP option checking. Next, in step 24 a packetheader classification control step is performed in which the packet issubject to admission/security control, packet prioritization, MPLSlabeling, tagging, resource re-configuration control (e.g.,buffer/network path resource management and reservation).

Packet classification is then performed as shown in step 26. In thisstep the packets are classified according to the function to beperformed and/or queuing priority as indicated by the header destinationaddress, priority/restriction designation. An example of functions to beperformed is those packet(s) to be dropped per the header tagindication. Next, in step 28 a packet forwarding process in which aforwarding table lookup step is performed for packet header update withnew destination port, TTL, etc.

The packet is then queued in step 30 per the classification made in step26 and designated destination port in step 28 and then, as shown in step32, the packet is scheduled per the classification. Next, the packet isswitched to and queued at the appropriate output port as shown in steps34, 36.

The QoS control plane functions are those functions associated with step24. Hence, “edge” routers must be able to perform steps 16 and 22–34(i.e. eight steps) for packet processing including step 24 for the QoScontrol plane functions, whereas “hub/backbone” routers need all butstep 24 in FIG. 2. In particular, the QoS control plane is responsiblefor but not limited to the setting of policies and rules for packetclassification control, admission control, and network resourceadministration.

Referring now to FIG. 3, a portion of a Baseline Reconfigurable LimitedScalable (BRLS) router 40 includes a router special need agent (RSNA)42. The RSNA 42 is a hardware component that is part of the routerarchitecture that takes on functions of not only a Quality-of-Service(QoS) control plane, but also some data plane functions. Communicationbetween the RSNA and other portions of the router is viarequest/response messages 44.

It should be noted that the RSNA 42 is part of the router's internalfunctional units. Other descriptions on possible implementation of theRSNA for a BRLS router are provided below. The BRLS router includes oneor more general purpose IM receivers 46 each of which includes an IM orline card 48. Each of the line cards 48 are coupled to (or incommunication with) the RSNA 42 via request/response messages 44. The IMreceivers 46 are also coupled to a network interconnect module (NIM) 50and a network processing module (NPM) 52. The NIM 50 is the router'sswitch fabric and the NPM 52 is the router's network processor (or CPU).The RSNA 42 acquires topology information from the NPM 52.

A craft GUI 56 is coupled to the NPM 52. Through the craft GUI, newalgorithms or new administrative roles/policies can be rapidly installedinto the appropriate functional units to adapt to changing features.

Requests to the RSNA 42 are in the form of headers of packets that needto have “classification tags” according to its specific requirements.Similarly, response from the RSNA 42 is the “newly tagged” updatedpacket header itself.

Classification tags are assigned, for example, by matchingQoS/Class-of-Service (CoS) information, e.g., the TOS—Type ofService—byte, or the 5-tuples: source/destination IP address,source/destination port number, and protocol, indicated in the packetheader according to the RSNA's internal data structures onclassification rules and policies.

Request/response sessions to the RSNA are triggered whenever an“edge/ingress” router receives a packet with a header indicating specialservice requirements that can't be fulfilled by processing througheither the router's data plane or the router's central processing unit(CPU). The responses are preferable in the form of already updatedpacket headers. Thus, the router itself further processes the header byperforming table lookup, integrating it with its payload and thenqueuing the packet according to updated “instructions” in the header.

Hence, in addition to administering and provisioning policy and control,the RSNA 42 actually must identify/sort out individual headers ofpackets according to their special requirements and “tag” each accordingto its internal policies/rules, etc. that meet the requirement. Thepurpose of executing these data plane functions in the QoS control planeis to reduce the data plane functionality (in the router) to a “bareminimum” thereby achieving the uniformity of only having genericfunctions in the data plane for all routers, whichever kind of router itis: “edge”, “hub” or “backbone”. This thus provides one importantconcept to hardware reconfigurability: a unified internal hardwarearchitecture of the “Methodology for Reconfigurability” as that term(i.e. the term reconfigurability) is explained in the above-identifiedTse-Au thesis. For example, with RSNA taking on packet header updatingfunctions, sophisticated packet classifiers are no longer needed, butrather only simple classify function suffices in the data forwardingpath within any router, where speed is of the essence.

In one embodiment, every router in the network has a connection to theRSNA. The RSNA function/connection is always ON for a router designatedas “edge”, but may be OFF for a router designated as either “hub” or“backbone”, depending on implementation. One example of implementationthat requires this connection to be ON for a router designated as “hub”or “backbone” is in I. Khalil, T. Braun, “Implementation of a BW Brokerfor Dynamic End-to-End Resource Reservation in Outsourced VirtualPrivate Networks”, Proceedings, 25th Annual IEEE Conference on LocalComputer Networks, LCN 2000, pp. 511–519.

As mentioned above in conjunction with FIG. 1, configurability (orreconfigurability) refers to the ability of a router to be(re-)configured “downward” as “edge” or “upward” as “hub/backbone”router. “Bi-directional configurability” (or “bi-directionalreconfigurability”) refers to the ability of a router which is firstconfigured as an “edge” to then be configured as a “hub/backbone” andthen again as an “edge”. The reverse is also true, that is, the routermay first be configured as a “hub/backbone” and then “configured” (or“re-configured”) as an “edge”. This ability to be repeatedly configuredin either of two directions is referred to herein as“bi-directionality.” This ability immediately follows with the on/offflexibility provided by the RSNA. Thus, the router can be versatiledepending on its physical placement within the ISP network hierarchy.

Referring now to FIG. 4, a Baseline Reconfigurable Limited Scalable(BRLS) router 70 includes an interface module (IM) receiver 72 whichcontains an Interface Module (IM) 74 (i.e., the router's networkinterface or input/output line card); one or more (centralized) NetworkProcessing Module(s) (NPMs) 76; a CoS/QoS preserving NetworkInterconnect Module (NIM) 78; and a Router Special Needs Agent (RSNA)80. The IM 74 and the RSNA 80 may be similar to the IM 48 and RSNA 42 ofFIG. 3.

It should be appreciated that a BRLS router can be bi-directionally(re-)configured as “hub”, “edge”, or “backbone” router as desired, forexample, during network hardware expansion/upgrade or emergencysituations via use of not only the RSNA but also the general-purpose IMreceiver 72. Being general-purpose, this receiver is identical in allrouters and allows IMs of any functionality (be it “edge”, “backbone” or“hub”) to be operationally compatible when inserted into the receiver inany BRLS router.

All internal functional components of the BRLS router such as the logiclevel chips within the IM, are interconnected together via a high-speedelectronic connection. It should be appreciated that the particularembodiment of the BRLS router shown in FIG. 4 does not include anyoptical components although in some applications, it may be desirable oreven necessary to provide the BRLS router having at least some opticalcomponents.

Likewise, the interiors of each functional component within the RSNA,e.g., logic level chips as well as all channels interconnecting thefunctional components, are connected together via high-speed electronicconnections. It should, however, be appreciated that in someapplications it may be desirable or even necessary to provide the RSNAhaving at least some optical components. As the IM receiver is identicalin all BRLS routers, the difference in throughput between an “edge” anda “hub/backbone” router is achieved using a modular interconnection.That is, an “edge” router will have a smaller number of IMs/IM receivers(thereby curtailing the amount of throughput) than a “hub/backbone”router.

As shown in FIG. 3, request/response messaging actually takes placebetween each individual IM 48 (of the BRLS router) and the RSNA 42. Asalso shown in FIG. 3, there exists a craft Graphical User Interface(GUI) 56 at the router's NPM 52.

In E. S. H. Tse-Au, “Design and Analysis of Scalable Reconfigurable IPRouters”, Ph.D. Dissertation, Dept. of Computer Science, StevensInstitute of Technology, Nov. 19, 2001, three candidate switch fabrics(SFs) are found to be capable of serving a scalable bi-directionallyreconfigurable IP router. These switch fabrics are: hypercube, 3D TorusMesh, and Multistage Interconnection Networks (MINs).

Any of the above-three candidate SFs can be used in describing thehardware architecture of the present invention and two of the three SFs:(hypercube and 3D Torus Mesh) belong to a common class ofInterconnection Networks (INs) called k-ary n-cubes. It should thus benoted that a k-ary n-cube was arbitrarily selected as the SF in thecomponent descriptions provided herein below. Those of ordinary skill inthe art should appreciate of course that a k-ary n-cube, a MIN or anyswitch fabric having the following characteristics could be used.

-   -   Shared (vs. dedicated) packet transfer hardware paths amongst        all packets, but most importantly,    -   Ease of incremental (hardware) expandability/contractibility        with respect to the incremental space and/or incremental        capacity requirements required on demand. Incremental capacity        requirement refers to the notion that only the incremental        amount of network capacity (e.g. number of hardware components,        associated bandwidth requirements) needed to accommodate the        incremental expansion at the moment must be considered in the        expansion. A router hardware architecture that fulfills this        requirement is one the existing infrastructure of which before        the expansion doesn't need to already have the capacity desired        at the maximum expanded configuration be pre-provisioned.        Whereas incremental space requirements refer to the notion that        only the required physical space be just enough to accommodate        the incremental capacity needed for expansion at the moment must        be considered.

The BRLS router of FIG. 4 includes an intelligent re-programmablehardware-based forwarding engine a portion 84 of which is shown in FIG.4. The forwarding engine is capable of forwarding and switching via theuse of re-programmable hardware-based functional units, and also has oneor more (software-based) processors to perform miscellaneous,non-time-sensitive tasks such as traffic statistic compilation andaccounting. It performs all physical layer/MAC layer functions,functions for receiving/transmitting/processing ATM cells of fixed size,and multiprotocol data frames of any size. The forwarding engine alsohas primary responsibility for the distributed processing and forwardingof all packets going through the router. Depending upon its nature,there are in general three types of packets as follows.

A first type of packet is known as a “regular” packet. “Regular packets”are those packets with no errors/IP options, or those not requiringspecial services such as priority queuing, packet drop, admission policycontrol, etc. This type of packet requires processing only throughfunctions within the router's data plane.

A second type of packet is known as a “non-time-sensitive” packet.“Non-time-sensitive” packets are those packets requiring attention ofthe NPM (i.e. packets going to the NPM). Such packets include, but arenot limited to, packets with errors (so-called “errored packets”), IPoptions, those destined to the router itself (e.g. control packets) orthose having an address which could not be found in the Tables Lookup ofthe router.

A third type of packet is known as a “special needs” packet. “Specialneeds” packets are those packets with special processing requirementssuch as multicasting, broadcasting, MPLS, priority classificationcontrol, policy control, etc.

Using the above background, an overall view of packet processing throughthe IM is provided. As shown in FIG. 4, upon entry to the router IM forOpen Systems Interconnection (OSI) Level 3—Network Layer processing 85,a packet is stripped of its header 86 and the header goes through theinitial packet filtering 88 and/or Tables Lookup functions 90 while thepacket payload is stored in a buffer 87. Based on result of this initialheader filtering, packet header processing is separated into threedifferent “logical” processing paths: a “time-sensitive” path 92; a“non-time-sensitive” path 94; and an RSNA “special needs” paths 96.These paths are briefly described as follows:

(a) the “time-sensitive” logical processing path is the path thatphysically comprises all functions in the data plane. As will becomeapparent from the description below of the “non-time-sensitive” and“RSNA” logical processing paths, the “time-sensitive” logical path isused by all three types of packets mentioned above. However, a vastmajority of the packets that will be using this path are “regular”packets, since they don't require any special processing beyond thatprovided by the processing functions in the router's data plane. For“regular” packets, upon receipt of an output port address from TablesLookup in the “time-sensitive” logical processing path, headers of these“regular” packets join their own payload at the Header/PayloadIntegration Unit 100. “Whole” packets then go through the rest of thedata plane functions in the “time-sensitive” logical processing path tobe journeyed through the switch fabric (after packet queuing 102 andscheduling 104) to the intended output port.

(b) the “non-time-sensitive” logical processing path is used forprocessing errored packets and other packets requiring the attention ofthe NPM as well as those packets whose destination addresses are notfound in the Tables Lookup. The “non-time sensitive” logical processingpath for the errored packets and those other packets destined to the NPMitself is different than the “non-time sensitive” logical processingpath for packets whose destination addresses are not found in the TablesLookup function. Considering first those packets destined to the NPMitself (e.g. control packets), after passing through the packetfiltering function 88, stripped headers on the “non-time sensitive”logical processing path 94 rejoin their payloads at the Header/PayloadIntegration Unit 100, and from that point on the rejoined “whole”packets go through the rest of the data plane functions for transfer tothe NPM.

Considering next those packets with headers which contain destinationaddresses not found in the Tables Look-up Unit, after passing throughthe packet filtering function and the Tables Lookup function only thepacket header will be forwarded to the NPM. The NPM will be described indetail further below. Suffice it here to say that the NPM has a mastertable of addresses and for packet headers with destination addressesthat couldn't be found in the IM's forwarding table, the NPM doesaddress lookup, updates the header and then sends the header back to theTables Lookup unit of the originating IM to be processed again. Thepacket header is sent to the NPM by “skipping” over the Header/PayloadIntegration Unit 100 via path 93. Payloads of the headers being sent tothe NPM for address update remain in the input buffer 87 awaiting returnof the headers. Thus, those packet headers with destination address notfound in Tables Lookup are sent through the logically “slow” path to theNPM for address lookup and the input buffer stores payloads of thesepackets until the updated header returns from the NPM to the TablesLookup.

Upon return from the NPM on path 95, the updated packet headers gothrough Table Lookup 90 in the “time-sensitive” logical processing pathto get output port assignments. Once the port assignments are provided,the corresponding payloads are sent to the Header/Payload IntegrationUnit on path 97 and the header rejoins the payload at the Header/PayloadIntegration Unit 100. From that point on, rejoined “whole” packets gothrough the rest of the data plane functions in the “time-sensitive”logical processing path to be journeyed through the switch fabric (afterpacket queuing 102 and scheduling 104) to the intended output port.

(c) The RSNA path is used for processing packets the headers of whichindicate “special needs.” Packet headers that are processed through thispath journey to the RSNA for packet header “tag update” via therequest/response communication session. Upon return from the RSNA 98,the updated header then goes through the rest of the data planefunctions in the “time sensitive” logical processing path starting withTables Lookup through it's journey across the switch fabric (afterpacket queuing 102 and scheduling 104) to the intended output port.

In view of the above, it should thus be appreciated that while thepacket headers are separated into different logical paths for processing(e.g. the three logical paths described above). However, once updatedheaders return from all three paths, they will rejoin their respectivepayloads and the “whole” packets go through the rest of the“time-sensitive” logical processing path. In other words, all packets(except for those destined for the NPM itself) eventually go through the“time sensitive” logical processing path. The overall effect of thispath separation for processing headers with different needs is,hardware-wise, a much simplified and speedy data plane for processingall packets passing through the router.

Part of the IM is a media/speed neutral general-purpose IM receiver thatsupports all interfaces and speeds which a BRLS router supports. Oneexample of a general purpose IM receiver is Juniper router's IM receiverin its M20 gigabit edge/backbone routers and M40 gigabit backbonerouters, respectively capable of 20 Gbps and 40 Gbps aggregated speed.The IM receiver design for these two routers are exact replica of eachother such that all physical IMs supported by the two routers can beinterchangeable in the two routers' chassis.

For the IMs in the BRLS router, the IM receivers will have N:1 sparring(i.e., there's one hot standby for every N IM receivers) and an IM willbe hot swappable.

To compliment the general-purpose IM receiver concept, IMs that are“fully capable” are necessary to serve the “edge”, “hub” and “backbone”function in an IP network. “Fully capable” is the ability to do at leastthe following: (a) adaptation to a multiplicity of connectiontechnologies; (b) channelization to a multiplicity of speed requirementsunder the multiple connection technologies (e.g., GE, DSL); and (c)multiplexing/de-multiplexing.

At this point in time, the aggregate speed of channelization for “edge”routers is at most OC12 but the current industrial trend is to increasethe aggregate speed to OC192 while retaining the channelizationcapability over the OC192→fractional T1/DS0 spectrum, and flexibleadaptation to heterogeneous connection technologies. Hence, the notionof “fully capable” IMs will continue to improve as channelizationtechnology under multiple connection types matures over the next fewyears.

Router architectural scalability is addressed by complete functionalpartitioning of all functions for the packet-forwarding path within theforwarding engine. Examples are filtering, table lookup, queuing,scheduling. Each of these functions has its own dedicated datastructures and memory capacity needed to perform its specific operation,is implemented as hardware-based algorithms using (re-)programmablehardware such as Field Programmable Gate Arrays (FPGAs) andre-programmable Application-Specific Integrated Circuits (ASICs), and isarranged in pipeline fashion for efficient packet/header processing. Inpreferred embodiments, a craft GUI interface is utilized to (re-)programas needed all of the (re-)programmable hardware-based functions to adaptto the dynamic needs of a network (e.g. an IP network) to respond to thecontinuous network feature changes and traffic growth demand.

The header/payload integration unit 100 is not part of the fundamentalpacket-forwarding functions, but can also be implemented via hardwarebased algorithms. The integration unit performs two major operations:(a) integration of updated packet headers (with the new routinginformation) back to the corresponding packet payload; and (b) based onthe new routing information, segmenting an entire packet into fixed-sizeunits and assigning each fixed-size unit a packet identifier (for packetre-assembly purposes) in preparation for packet queuing and schedulingfor transfer through the SF.

In case of multicasting, the RSNA 80 is responsible for producingmultiple packet headers needed for the different destinations in amulticast session. Whereas the header/payload integration unit isresponsible for producing the corresponding number of packet payloads106, keeping track of and integrating each payload to its correspondingupdated header when it arrives back from the RSNA. When the new routinginformation for the packet is obtained during integration, theheader/payload integration unit then treats each packet in a multicastsession as a normal packet. It segments each into fix-sized unit,assigns each unit with packet identifier based on the new routinginformation, and queue the fix-sized unit per this identifier forpreparation to transport through the SF for the multiple destinations inthe multicast session. The exact details of how the two main operations(listed above) are done or the size of the fix-sized unit, isimplementation dependent.

The output link controller 108 is responsible for scheduling the use ofoutput links (or uplinks) to output packets from various input(down)links. To address the anticipated need for assigning packets withpossibly a wide range of incoming speeds to uplinks that also differwidely in BW capacities in future IP traffic environments, a briefoverview of the scheduling method used by this output link controller isprovided. For the following discussion, it is assumed that each input(down) link will not carry input traffic with speed greater than (>) thenominal bandwidth (BW) of that input link. However, input (down)linkscan carry input traffic with speed less than (<) its nominal BWcapacity. As a general design rule-of-thumb a downlink should always beassigned an uplink the BW capacity of which is at least equal to itself,not lower, unless: (a) the incoming packet speed from this downlink is“detected” to be ≦ the nominal BW capacity of an uplink that has a lowernominal BW capacity than that of this downlink; or (b) the incomingpacket speed from this downlink is “detected” to be > the nominal BWcapacity of an uplink that has a lower nominal BW capacity than thisdownlink nominal BW capacity, and there exists a network emergency, suchas when the current uplink being used is dead and no other uplink ofcomparable speed is available.

Hence, to accommodate congestion or unforeseen network events, there arein general two assignment methods an output link controller can use: (a)when all uplinks have similar or higher nominal BW capacity than any oneof the downlinks, assign each downlink a designated primary andsecondary uplink. Then assign an output packet to its secondary uplinkif its designated uplink is congested or malfunctions. This method ofassignment induces the effect that all uplinks are used in parallel, sothat the net achievable theoretical output is the aggregate BW capacityof all the uplinks during normal operation; and (b) when ≧1 uplinks havelower nominal BW capacity than ≧1 downlink(s)', assign an output packetto an uplink based on packet's incoming traffic speed or incoming linkBW capacity as follows: (1) if the downlink nominal BW capacity fromwhich the packet came is ≦ any of the output (up)link's nominal BWcapacity, assign the packet from this downlink as in Method (a); else (∃uplink(s) the nominal BW capacity of which is < this downlink's own),then assign an output packet to an available uplink the nominal BWcapacity of which is ≧ the packet's “detected” incoming traffic speed(for this case, ∃ no concept of primary or secondary uplink).

This method of assignment induces the net effect that lower speed inputtraffic will go to lower nominal BW capacity output (up)links, includingthe “lower” speed packets that came in through a high nominal BWcapacity input (down)link; and higher speed input traffic will go tohigher nominal BW capacity output (up)links during normal operation.

It should be noted that the scenario described in method (b) above ispossible in a router with downlinks that vary widely in speed limits(e.g., DS0 and OC192) and with uplinks that are of comparably largedifference in speed range. In such scenarios, lower speed input trafficis assigned an output (up)link of relatively lower BW capacity andhigher speed input traffic is assigned an output (up)link of relativelyhigher BW capacity under normal operations. Of the two, method (a) iscurrently the norm as the BW capacity of uplinks today is at least thesame as the BW capacity of any one downlink within the same router.

The set of output buffers 108 include packet reassembly buffers forre-assembling each segmented packet back together, re-sequencing buffersfor the purposes of packet re-ordering to meet in-order deliveryrequirement, and smoothing buffers to meet QoS provisioning needs,before transporting packets into the output link. These output bufferstemporally hold not only (reassembled) packets that are destined to theoutput, but also “transit (segmented) fix-sized units” of packets thatcome from some IMs and destined to other IMs. This is due to the factthat all IMs serve as intermediate IMs for some (segments of) packets ina k-ary n-cube SF. Within an intermediate IM, a routing decision must bemade to select the (next channel or) next IM to route the (segmented)packet to as part of its journey to destination IM. These “transitfix-sized units” go through table lookup to make a routing decision asto which channel to route to in its next hop to destination IM. Thistype of table lookup would be based on header “tags” and thereforerequire the availability of only simple routing/classificationinformation to perform the routing task. Upon completion of tablelookup, these “transit fix-sized units” go through queuing andscheduling again for switching through the k-ary n-cube SF.

The RSNA 80 is an independent functional component that is part of theinternal router architecture. It is responsible for the CoS/QoS ControlPlane, and some amount of data plane functions, and provides packetswith resource and reservation information pertaining to hop-by-hop viewof the network.

One example of a data plane function is the production of multiplepacket headers needed for the different destinations in a multicastsession at the receipt of an IP packet header having a multicastrequirement. As previously described, the RSNA communicates with the IMsvia request/response messages. The “request” being a packet header thatneeds to be “tagged”; and the “response” being the “newly tagged”updated packet header itself.

The RSNA 80 coordinates with the Network Processing Modules (NPMs) 76,to obtain updated network topology information for use with itsrules/policies. Communication with the NPM could be via file transferformat, to accommodate large topology tables that may be required attimes, or could be via request-response format if small amount ofinformation exchange is involved, or both, depending uponimplementation. To induce parallelism of operation, packet headers withdifferent QoS/CoS policy requirements are processed in different queues.Within each queue that processes headers of packets with similarrequirements, all functional units, i.e., algorithms such as tablelookup and scheduling per the QoS/CoS requirements indicated in theheader, are (re-)programmable hardware-based and are arranged inpipeline format for efficient packet header processing. Furthermore,these internal functions are interconnected to one another viaelectrical channels.

As the internal algorithms are re-programmable hardware-based, the craftGUI interface 82 through the NPM 76 is available for re-programmingpurposes. This interface 82 is also used to (re-) configure newpolicies/classification rules to the RSNA's internal data structures toaccommodate the increasingly sophisticated and dynamic QoS provisioningneeds over IP.

The Network Processing Module (NPM) 76 is a control engine primarilyresponsible for master route table maintenance, “external” routingprotocol operations with other routers within an IP network, “internal”routing protocol operations of the SF, and processing ofnon-time-sensitive/exception condition tasks, e.g., management and errorprotocol operations. Examples of “external” routing, error andmanagement protocols processed by the NPM(s) are ICMP, RIP, OSPF, BGP,SNMP, ARP, RARP, etc. (The type of internal routing protocols varieswith the type of SF used.) The NPM doesn't perform any packet forwardingfunction. As mentioned above, for packet headers with destinationaddresses that couldn't be found in the IM's forwarding table, NPM doesaddress lookup, updates the header and then sends the header back to theTables Lookup unit of the originating IM to be processed again.Moreover, the NPM provides a craft GUI for router management,maintenance and programming interface to (re-)program, i.e., (re-)configure, all hardware (either ASIC and/or FPGA)—based algorithms ineach IM and in the RSNA. Examples of hardware-based algorithms arepacket filtering, packet queuing based on tags, media-specificalgorithms, table lookup, and configuration of new policies/rules (thislast item applies for the RSNA only, not the IM). As the router NIM isall electrical, the NPM has a corresponding electrical SF channelinterface.

Other tasks that NPM performs are: (a) notification and distribution ofupdated network topology, multicasting, unicasting, other (hop-by-hopview) path information to RSNA; (b) packets with IP options; (c) framerelay mapping operations; (d) distribution of updated route tables for“external” IP network routing to the IMs; (e) distribution of updatedroute tables to the IMs for “internal” SF routing (e.g., the best routesusing Wormhole routing to get from a source to a destination IM to avoidfaulty conditions within the k-ary n-cube SF); and (f) distribution of(new) programs to each hardware-based algorithm within each IM and theRSNA;

It should be noted that ARP/RARP/Frame Relay Mapping operations are notconsidered time-sensitive protocol operations in this proposal and thusare processed by the NPM. Packets with these needs are in the minorityand the implementation of their operations and associated lookup tablesat each IM (i.e., at the time-sensitive path) may delay other“time-sensitive” operations that must get done within the IM.

Due to the non-time-sensitive nature of task processing by the NPM, ageneral-purpose CPU can be used to satisfy its processing needs. Toaddress the need for redundancy in case of failure, there will be asecond active standby NPM within the NIM infrastructure (describedbelow).

The network interconnected module (NIM) 78 is an interconnectioninfrastructure that connects amongst the intelligent IMs and NetworkProcessing Modules (and also the RSNA in this architecture) viabi-directional electrical channels. Using K-ary n-cube as the SF, theNIM is capable of incremental hardware expandability and hence IMs canbe added to scale according to traffic capacity growth. The NIM'sresponsibilities include: (1) transfer of (segments of) “regular”packets, i.e., those with no “special needs” and “special needs” packets(that are already “tagged” by and have returned from the RSNA) betweeninput IM and output IM (these two types of packets can therefore beprocessed via the “time-sensitive” path); (2) transfer of packet headerswith “special needs” to the RSNA; (3) transfer of (segments of) controlpackets or (segments of) packets between IMs/RSNA and the NPM(s), e.g.,those that can be processed via the “non-time-sensitive” path, such as(updated/new) programs to (re-)program the (re-) programmablehardware-based functions within the IMs and the RSNA and updated routinginformation (external or internal) to the IMs; and (4) occasionaltransfer of packet headers (either “regular” or “special needs”) betweenthe IMs and NPM(s) for which the destination address could not be foundin the forwarding table.

It should be appreciated that all connections between RSNA and NPM(s),and between RSNA and IMs are actually part of the NIM. However, as FIG.4 is a functional diagram, they are drawn away from the NIM to clearlyillustrate their functional interworking relationship.

Provided below is a description of how the BRLS router hardwarearchitecture provides for (1)hardware scalability to grow and adjustaccording to the network capacity growth demand with flexible adaptationto changing IP network features; and (2) bi-directional hardwarereconfigurability to perform different roles depending on router'sphysical placement in the ISP network hierarchy: “edge”, “backbone” or“hub” layer.

In particular, an explanation is provided for the latter how thissolution can be used for efficient hardware expansion and feature/speedconversion, such as during network hardware expansion/upgrade andemergency network rescue situations.

(1) Hardware Scalability: The BRLS router described above is readilyimplementable with the current state-of-the-art technology. Within theBRLS framework, the ability to accommodate a feature-rich set of QoScapabilities and with increasing throughput requirement can be addressedby use of reprogrammable hardware-based functional units in the BRLSrouter and in the associated RSNA. The reprogrammable hardware-basedfunctional units enable reprogramming of up-to-date entries to“appropriate” tables and algorithms for fast table search to provideappropriate QoS features. Scalability to meet increasing throughputdemands is further provided by the partitioning and pipelining of each(re-)programmable hardware-based function used in the packet (or header)processing paths within the BRLS router and the associated RSNA. Forexample, with a full pipeline, one task per CPU cycle could be achieved.This arrangement allows parallelism of operation, divides and conquers,and thus enables the BRLS router and the associated RSNA the ability toprocess traffic efficiently at very high speed in a traffic-volumeindependent manner.

As the SF to be employed by the BRLS router is capable of incrementalhardware expandability (k-ary n-cube or MIN—Multistage InterconnectionNetwork), the BRLS router also has the HW flexibility to add I/O portsand adjust according to traffic capacity growth demands. Moreover,reprogrammability of the individual functional units within the routerand within the associated RSNA provide for flexible adaptation to theconstant changes in QoS/CoS provisioning needs along with exponentialtraffic growth within an IP network.

Per the above description of the packet-forwarding path's setup withinthe BRLS router and the RSNA component, and the notion that BRLSutilizes SFs capable of incremental hardware expandability, hardwarescalability to high traffic volume ensues in the data plane. However,even though the RSNA by itself can handle a large volume of traffic, thefact that it is part of the router internal architecture limits therouter's QoS control plane scalability.

As IP networks evolve to provide continuously more QoS/CoS-based serviceofferings that require overall network view information, a built-in RSNAfunction as part of the router simply will not be able to scale in termsof expansion, to the whole path data storage capacity needed to meetthis demand. Only a hop-by-hop non-optimal view of network resourcecapability could be offered in this built-in configuration, and hencethe need for global synchronization of state among routers in thenetwork. A built-in RSNA function also increases the implementationcomplexity, as RSNA must be integrated/evolve with other router internalfunctions. For example, in the case of faster RSNA QoS/CoS table lookupalgorithms, once one router upgrades to it, then all routers in thenetwork must upgrade to it in order to be able to communicate with oneanother. Any policy change will also involve effort to upgrade allrouters in the network.

(2) Bi-directional Reconfigurability: Bi-directional reconfigurabilityis achieved with the on/off flexibility provided by every router'sconnection to the RSNA. This bi-directional reconfigurability can beused either during network hardware expansion/upgrade or emergencysituations. During emergency situations, the bi-directionalreconfigurability is used to turn “edge” routers into “hub/backbone”routers by clustering together several “edge” routers to meet thethroughput requirements of a “hub/backbone” and to turn “off” the RSNA(as called for by the implementation) and to have “Hub/backbone” routersact as “temporary” edge routers by simply turning “on” the RSNA.

During a network hardware upgrade/expansion operation to increasetraffic throughput for “Hub/backbone” routers to automatically becomenew “edge” routers is accomplished by turning “on” the RSNA, providingthe quickest provisioning turn-around time. “Edge” routers become“hub/backbone” routers by adding more IM receivers, IMs and channels,and by turning “off” the RSNA (as called for by the implementation). Itshould be noted that the limit on the top number of receivers is boundedby engineering choice in practice. Moreover, if k-ary n-cube is used asthe SF, the above addition can be made possible only if the existing IMsand channel wires in the existing NIM infrastructure have enoughcapacity ready for expansion when first implemented. This means thatexisting IMs should have a sufficient number of interface pin outs andexisting channels should have sufficient BW capacity to cope with theincreased traffic volume resulting from an increased number of IMtraffic sources.

Referring now to FIG. 5, in which like elements of FIG. 4 are providedhaving like reference designations, a Baseline Reconfigurable LimitedScalable (BRLS) router is shown. The router is similar to the BRLSrouter described above in conjunction with FIG. 4 except that in thisembodiment, the switch fabric used for the NIM 78 is implemented as amultistage interconnection network (MIN). As mentioned above, the MINswitch fabric is capable of serving as a scalable reconfigurable (andbi-directionally reconfigurable) router including a scalablereconfigurable IP router.

It should be noted that when the k-ary N-cube switch fabric is used,transit traffic propagates between the NIM 78 and the output linkcontroller and buffers 108 in each IM. When MIN switch fabric is used,however, no such path exists. Thus, one difference between the k-aryn-cube and BMIN SFs is that no transit traffic is possible through theIM, as all transit traffic visits the intermediate switching elements(SEs) only. Consequently, output buffers in the IM are not used tobuffer any (segments of) transit packets in a BMIN SF.

It should be appreciated that the functional description andarchitectural layout for the BRLS architecture using the k-ary n-cube SFdescribed above in conjunction with FIG. 4 in general applies to thatfor the BRLS using the bi-directional MIN (BMIN) SF shown in FIG. 5.Differences are explained below.

The NIM is a bi-directional MIN consisting of the N×N entities that areIMs, NPM(s) and RSNA, interconnected together via O(N log_(b) N)intermediate switching elements (SEs) and O(N) bi-directional electricalchannels. The SEs are implemented with (re-)programmable hardwarecomponents such as ASICs or FPGAs.

It should be noted that the number of IMs/NPM(s) residing in each of theN×N entities, as well as the BMIN interconnection pattern isimplementation dependent. Moreover, the NIM can use more than one BMINto provide QoS/CoS and fault-tolerance. Other functional descriptions ofthe NIM for a BRLS router using k-ary n-cube SF hold for the BRLS routerusing BMIN SF.

The functions of packet queuing and scheduling are also needed insidethe IM rather than relegating them entirely to the SEs, to exercise theprinciple of divide and conquer as explained in the Tse-Au thesisreferenced earlier. Packet queuing and scheduling inside the IM is toaccommodate the possibility of QoS/CoS provisioning needs for (some)packets, whereas packet queuing (and any scheduling) inside SEsaccommodates the case of internal blocking and path sharing that arecharacteristic of some types of BMINs. The result of applying the divideand conquer principle greatly simplifies the implementation and packetswitching process of the SEs. For example, SEs no longer need to performinitial packet QoS prioritization based on policies/rules. This lessensthe delay of this potential switching bottleneck in a packet's path todestination.

The set of output buffers within an IM includes reassembly buffers toreassemble segmented packets, and packet smoothing buffers to meet QoSprovisioning/service level agreements (SLA) guarantee requirements.Moreover, it also includes packet re-sequencing buffers to accommodatescenarios in which packets can be delivered OOO due to techniquesemployed to overcome hot spot traffic conditions in MINs, such as duringuplink access.

Output link schedulers are needed in each IM to properly schedulepackets to available output links during network congestion or emergencysituations.

Based on the above descriptions for the IM, all IM architecturalcomponents remain the same as used for the case when the NIM is a directnetwork k-ary n-cube. Thus, the principle of hardware reconfigurability(availability of a unified internal hardware architecture) has beenapplied to achieve a unified internal hardware architecture for the IM,thereby resulting in a reconfigurable IM functional architecture in anylevel of the network hierarchy capable of interconnecting to eitherdirect or indirect NIMs.

Assuming self-routing property, NPM(s) need not distribute an internalSF routing table.

It should be noted that in FIG. 5, all connections between the RSNA andNPM(s), and between RSNA and IMs are actually part of the NIM. As FIG. 5is a functional block diagram, they are drawn away from the NIM toclearly illustrate their functional interworking relationship.

Referring now to FIG. 6, a network includes a plurality of user nodes130 a–130N coupled via a local area network (LAN) 132. The LAN 132, inturn is coupled to an “edge” router 134 which receives packetstransmitted thereto. The “edge” router is coupled to a “hub” router 136which is in turn coupled to a “backbone” router 138 which provides apath to the Internet 140. Each of the edge, “hub” and “backbone” routersis coupled to an external router special needs agent (RSNA) 142. Thus,there is a connection to the RSNA from each router in the network (e.g.via request/response sessions 144–148 with the IMs of each of therespective routers). A router architecture which utilizes an externalRSNA is referred to herein as a one-architecture-fit-all scalablereconfigurable router (OSR).

The RSNA function is always on for an OSR router designated as “edge”,but may be off for an OSR router designated as either “hub” or“backbone.” Bi-directional reconfigurability immediately follows withthis on/off flexibility to the RSNA, which can be used either duringnetwork hardware expansion/upgrade or emergency situations.

For example, if a “hub” or “backbone” router unexpectedly stopsoperating, the network hierarchy can be re-configured such that a nearby“edge” router can perform the functions of the “hub” router by turningoff the RSNA connections as called for by the implementation. Theconverse is also true, that is, if an “edge” router unexpectedly stopsoperating, the network hierarchy can be re-configured such that a nearby“hub” or “backbone” router can perform the functions of an “edge” routerby turning on the RSNA connections. This induces network reliability,provided that RSNA is also redundant.

Referring now to FIG. 7, an OSR router 150 includes a plurality ofgeneral purpose IM receivers 152 a–152N each of which includes aninterface module or line card 153 a–153N. The IM receivers and interfacemodules may be similar to the IM receivers and interface modulesdescribed above in conjunction with the BRLS router of FIG. 3.Differences between the OSR and BRLS routers are described below inconjunction with FIG. 8. The OSR router 150 also includes a networkprocessing module (NPM) 154 coupled to the IM receivers 152 a–152Nthrough an optical network interconnect module 156. It should beappreciated that the IM receivers 152 a–152N, the network processingmodule 154 and network interconnect module 156 are all physicallylocated inside the OSR router 150.

A router special needs agent (RSNA) 160 having an optical channelinterconnect is in communication with the interface modules 153 a–153Nof the IM receivers, the network processing module 154 and a craft GUI155. It should be appreciated that the RSNA 160 is physically locatedoutside the OSR router 150.

The RSNA's communication strategy with other network elements such asthe OSR router 150, is similar to that used in the BRLS architecture.For example, the RSNA communicates with the IMs of the OSR router viarequest/response signals. For communication with the NPMs of the OSRrouter (to acquire topology information), the communication could be viafile transfer, depending upon the particular implementation. Bydecentralizing the RSNA functionality as a separate independent networkelement, scalability of the QoS control plane functions is enhanced andthe implementation of the data plane functions in the routers aregreatly simplified. The end result is a speedy/simplified forwardingpath/data plane at the routers for processing all packets (including“regular” packets, packets requiring the attention of the NPM and“special needs” packets).

Flexible bi-directional reconfigurability can be achieved duringemergency situations for “edge” routers (to act as temporary“hub/backbone” routers) by clustering together several “edge” routers tomeet the throughput requirements of a “hub/backbone” and to turn offRSNA (as called for by the implementation).

Flexible bi-directional reconfigurability can be achieved duringemergency situations for “hub/backbone” to act as “temporary” edgerouters by simply turning “on” the RSNA.

Bi-directional reconfigurability can also be achieved during a networkhardware upgrade/expansion process to increase traffic throughput for“hub/backbone” routers to automatically become new “edge” routers byturning on the RSNA, providing the quickest provisioning turn-aroundtime.

Bi-directional reconfigurability can also be achieved during a networkhardware upgrade/expansion process to increase traffic throughput for“edge” routers to become “hub/backbone” routers by adding more IMreceivers, IMs and channels, and by turning off the RSNA (as called forby the implementation). It should be noted that the limit on the topnumber of receivers is bound by engineering choice in practice and thatthe above addition can be made possible only if the existing IMs andchannel wires in the existing NIM infrastructure have enough capacityready for expansion when first implemented. This means the existing IMsshould have sufficient number of interface pin-outs and existingchannels should have sufficient BW capacity to cope with the increasedtraffic volume resulting from an increased number of IM traffic sources.

Based on the above description, the RSNA's responsibility includes notonly the traditional set of QoS control and resource reservation such aspath selection/BW reservation, it also includes any special packetforwarding needs that require information on overall network “path” viewsuch as network-wide path resource reservation, capacity and path state,to enable meaningful services. Examples of the latter are MPLS,multicasting, VPN provisioning, and MPLS-enabled VPNs.

Like the BRLS router, a key requirement to the success of thisarchitecture is a general-purpose IM receiver that's identical in allrouters, which enables the OSR's versatility to serve as “edge”,“backbone” or “hub” routers as desired. As the receiver is identical inall OSR routers, the difference in throughput between an “edge” and a“hub/backbone” router is achieved using modular interconnection. Thismeans that an “edge” router will have a smaller number of IMs/IMreceivers (thereby curtailing the amount of throughput) than a“hub/backbone” router.

Referring now to FIG. 8, an OSR router 164 includes an IM receiver 166coupled to a bi-directional K-ary N-cube network interconnect module 168(i.e. a K-ary N-cube switch fabric is used) which in turn is coupled toone or more network processing modules 170 and other IM and forwardingengines 172. The IM receiver 166, network processing modules 170 andother IM and forwarding engines 172 may all be functionally similar tothe IM receiver, network processing modules and other IM and forwardingengines described above in conjunction with FIG. 4. It should thus beappreciated that the functional description of the IM for the BRLSrouter described above in conjunction with FIG. 4 applies to that forthe OSR router 164, with certain differences as will be described below.

The IM receiver 166, network processing modules 170 and forwardingengines 172 are each coupled to an RSNA 174 which is external to the OSRrouter 164. Thus, one difference between the BRLS router and the OSRrouter 164 is that in the OSR router the RSNA is not part of therouter's internal architecture, but is a completely separate networkentity. Since the RSNA is a completely separate network entity, IMs mustaccess it via an “external” communications link 178.

Another difference between the BRLS router and the OSR router 164 isthat in the OSR router, the NIM is optical and the IM interior iselectrical. Thus, there is an additional device 179 at the IM's opticalchannel interface to the switch fabric for signal conversion betweenelectrical and optical (E/O) signals In practice, vertical cavitysurface-emitting lasers (VCSELs) can be used to provide a low-costsolution to such conversions for optical-electrical interconnect.

The RSNA 174 is a distinct server/database entity responsible for theCoS/QoS Control Plane that dynamically provides packets with resourceand reservation information pertaining to overall network view, ratherthan hop-by-hop. The basic functionalities of the RSNA are as describedabove in conjunction with FIGS. 3–7. Some details of these functions,their associated architectures and the interworking of these functionsboth internal to the RSNA and with other router component(s) external tothe RSNA are described below and in conjunction with FIG. 9

Referring briefly to FIG. 9, an exemplary RSNA 180 illustrating theparallel pipelines for the different types of packet headerclassification operations includes physical interface 182 through whichheaders of packets (or in alternate embodiments, entire packets) enterand exit the RSNA. The physical interface couples to a packet headerfilter 184 which performs packet header filtering. The packet headerfilter communicates with a plurality of separate/independent queues 186a–186 n and tables 188 a–188 n for separate types of QoS/CoS needs. Theexemplary RSNA of FIG. 9 is shown to include a multicast table, a packetdrop priority table, and tables for CoS/QoS admission, access control,and security admission. Although, several exemplary queues and tablesare shown in FIG. 9, it should be appreciated that the RSNA can includeany type and number of queues and tables required for any QoS/CoS need.

The multicast table 188 a leads to a header reproduction functional unit190 which in turn is coupled to an output buffer, queue and scheduler192 The tables 188 b–188 n are coupled directly to the buffer, queue andscheduler 192. The buffer, queue and scheduler 192 buffers, queues andperforms scheduling functions for newly tagged outgoing packet headersand provides the headers to the physical interface 182.

Because the individual functions in the RSNA can be (re-)programmable, acraft GUI interface 196 is available for this purpose. Through the craftGUI, new algorithms (e.g., for individual server computational use) ornew administrative rules/policies can be rapidly installed into theappropriate functional components to adapt to the changing IP networkservicing features. In this particular example, the RSNA includes aplurality of information bases and servers generally denoted 194 whichwill be discussed below.

Referring now to FIGS. 8 and 9, the RSNA is responsible for admissioncontrol as well as for resource reservation/management, e.g., buffer(re-)configuration, path selection/reservation for MPLS trafficengineering, multicasting and customized VPN provisioning on demand.Also, for MPLS traffic engineering, RSNA is responsible for the majorMPLS control plane functions (and hence RSNA can be said to be equippedwith the MPLS traffic engineering control plane) as follows: resourcediscovery; path selection; and path management. For multicasting, theRSNA participates in routing, flow and reservation protocols, computesshortest path multicast trees/inter-domain multicast tunnels, performsadmission control, reserves/allocates network path resources, and setupand maintains multicast trees/tunnels among admitted hosts. The RSNA isalso responsible for policy administration for differentiated CoS,packet drop, RSVP, security/access control, scheduling, andprioritization of flows. Hence the information bases and servers 194must be able to aid in the enablement of such functionalities.

RSNA architectural scalability is addressed by complete functionalpartitioning of basic packet header classification/processing functions:header filtering, queuing per QoS/CoS/other “special needs”, tablelookup per QoS/CoS/other “special needs”, header reproduction formulticasting, buffering and scheduling. Each of these functions has itsown data structures and memory capacity and is implemented by(re-)programmable hardware-based ASICs or FPGAs Furthermore, asillustrated in FIG. 9 the functions are arranged in series as pipelinesif there exists inter-dependence among functions, or in parallel ifthere is no inter-dependence.

Since communication with the routers (and more particularly with the IMsof the routers) in the RSNA's network domain of responsibility is viarequest/response, a request message enters the RSNA with a packet headerneeding a “tag” and a response from the RSNA with an “updated” packetheader containing the needed “tag.” For the case of multicasting, theRSNA is responsible for reproducing all the headers with differentdestination addresses needed for a particular multicast session, this isdone by the Header Repro Function 190.

The link state information bases and servers 194 maintained at the RSNA,include but are not limited to a topology information base, a policyinformation base, a flow information base, a path QoS state informationbase, a multicast tree/state information base and control server, anMPLS Traffic Engineering Control Plane information base, node QoS stateinformation base, an admission control server, a QoS routing server, apolicy control server and an MPLS Traffic Engineering control server(e.g., for MPLS path computation).

Sample contents of the multicast tree/state information base includesinformation needed to establish multicast sessions for all routerswithin the RSNA's domain of responsibility, such as access control list,multicast tree/tunnel state information for the multicast groups thateach router in its domain is involved in.

All information bases contain up-to-date link/path/flow state and statusinformation, and a variety of policy information, such that they act asinformation input to the individual tables that ultimately provide“tags” to the incoming packet headers. For example, RSNA collaborateswith the NPM of each router within its domain of responsibility toobtain up-to-date topology information for path selection/setup. Theservers perform path computations based on information from appropriateinformation bases. Results of computation are also fed to theappropriate tables.

For the RSNA within the OSR architecture, all-optical channels are usedfor interconnecting all internal components such as the functional unitsand servers as follows. All (re-)programmable hardware-based electronic(FPGA or ASIC) chips containing functional units such as packetfiltering, scheduling, etc., along with the associated tables andbuffers (i.e., the queues 186 a–186 n) are arranged in ≧ one cardinterconnecting to one another within each card via electronic interchipinterconnect. An E/O device (e.g., a VCSEL) is used at each card/opticalchannel interface for interconnection to other components within theRSNA. All information bases and servers are also equipped with E/Odevices at their interfaces to the optical channels for connection toother components within the RSNA. Finally, redundancy is addressed via1+1 sparing arrangement with other RSNAs within the ISP network.

This architecture having all-optical interconnects for the interior ofboth the RSNA and the OSR router, is readily implementable with thecurrent state-of-the-art technology Within the framework of this routerarchitecture, network hardware expansion or upgrade—the ability toaccommodate a next more feature-rich set of QoS capabilities and withincreasing throughput requirement, can be addressed by a distributedtwo-part network solution. It includes the upgrade of the OSR router andthe RSNA with “appropriate” tables and algorithms for fast table searchto provide appropriate QoS features. Scalability of both networkentities to meet increasing throughput demands is further provided bythe partitioning and pipelining of the functionality and by the(re-)programmability of each hardware-based function used in the packet(or header) processing paths within the two separate network entities.This arrangement allows parallelism of operation and thus enables theRSNA and the OSR router the ability to separately scale efficiently tovery high speed.

The decentralization of the RSNA as a separate entity allows it toserve, and therefore scales to, an arbitrarily large number of routersin its network domain of responsibility. The upper bound of this numberis part of overall network planning, considering factors such as theexpected volume of “special needs” traffic from each router within thenetwork, the type of queuing disciplines used to provide differentiatedpacket header processing within the RSNA to ensure line rate packetprocessing at the originating routers, etc. This decentralization alsoenhances the scalability of the QoS control plane functions therebyenabling the capability to offer value-added services on OVERALL networkpath basis, keeping minimal to no QoS state information in the routers.The consequence is this architectural solution's ability to adapt to theconstant changes in “whole path” QoS/CoS provisioning along withexponential traffic growth within an IP network into the foreseeablefuture.

The functional description of a NIM for the BRLS router using k-aryn-cube SF applies to that for the OSR router using k-ary n-cube SF withtwo differences. (1) the RSNA is not part of the router's internalarchitecture for the OSR and (2) the NIM 168 is made of all opticalchannels (vs. electrical channels in the BRLS router). With respect tothe first difference, the RSNA is a completely separate entity andtherefore the NIM doesn't need to be responsible for internal transferof: (a) packet headers with “special needs” to the RSNA; and (b)(segments of) packets between the NPM and the RSNA such as updated/newprograms to (re-)program the (re-)programmable hardware-based functionswithin the RSNA.

Likewise, the functional description of an NPM for the BRLS router usingk-ary n-cube SF applies to that for the OSR router 164 with onedifference being that the NPM will have optical channel interfaces; andfor that interface, a device capable of converting optical signals toelectrical signals and vise versa (i.e., E/O device will be used at thatinterface).

Referring again to FIG. 8, craft GUI 176 external to both the router 164and RSNA 174 is provided to (re-)program/(re-)configure allhardware-based algorithms in the IMs 167 of the router through the NPM.As RSNA 174 is a separate network entity, it has its own separate craftGUI 196 (FIG. 9) not from the NPM for (re-)configuration of its internalhardware-based algorithms. The NPM 170 still communicates with the RSNA174 for network topology updates via a separate interface to thetopology information base 194 a (FIG. 9).

Referring now to FIG. 10, an RSNA 220 having an optical channelinterconnect 222 is coupled to an OptoelectronicOne-Architecture-Fit-All Scalable Reconfigurable (OOSR) router 224. Apacket 226 enters the router 224 and is detected by a speed sensor 228within an IM receiver of the OOSR router 224. The IM 229 includes firstand second paths 230, 232. The first path 230 corresponds to an “almostall-optical” path (also sometimes referred to herein simply as the“optical path”) and may be provided, for example, from one or morere-programmable optoelectronic hardware devices 231 (e.g. FPGAs, ASICs,etc. . . . ). As will become apparent in subsequent explanations, theonly non-optical components in this path are the opto-electronichardware devices. Each of the devices 231 are coupled via an opticalinterconnect 234. The second path 232 corresponds to an all electricalpath and may be provided, for example, from one or more re-programmableelectronic hardware devices 236. Each of the devices 236 are coupled viaan electronic interconnect 238. At the end of the second path is anelectrical-to-optical (E/O) converter 240. Both the first and secondpaths lead to an optical channel NIM 242 that in turn also connects toother OOSR IMs with dual hardware technology paths as described above

In operation, the speed sensor 228 detects the relative speed of the IMconnection. If the detected speed is below a certain “watermark” (e.g.,at or below OC 12), the IM receiver directs the packets to the opticalchannel NIM 242 via the second 232 path within the IM 229 and the E/Oconverter 240. If, on the other hand, the incoming packet speed is abovethe “watermark”, the IM receiver directs the packets to the opticalchannel NIM 242 via the first 230 path within the IM 229.

Since the almost all-optical path 230 is faster than the electrical path232, the routing speed through the IM is different depending upon thepath. Thus, based on sensor result, the IM receiver “adjusts” therouting speed of the packet within the receiver by sending the packetthrough either path 230 or 232 within the IM 229 according to packetincoming speed.

It should be noted that regardless of the path 230, 232 on which thepacket is routed through the IM, packets in each path 230, 232 willstill have the three logical processing paths (i.e. time-sensitive,non-time-sensitive or RSNA as described above) available for processingthe packet depending upon packet processing requirements as indicated byinformation in the header.

It should be understood that although the RSNA is here shown as anexternal RSNA, there may be some embodiments in which it would bedesirable to provide the RSNA as an internal RSNA while stillmaintaining the optical and electrical paths within the IM.

Referring now to FIG. 11, an Optoelectronic One-Architecture-Fit-AllScalable Reconfigurable (OOSR) Router Architecture using K-ary N-cubeswitch fabric includes a set of IM receivers 202, each containing aseparate IM 201 with a forwarding engine. The IMs 201 have primaryresponsibility for the processing of “regular” packets. It should beappreciated that FIG. 11 shows an overall view of the OOSR router(without the RSNA) using k-ary n-cube SF and two IM receivers 202.

The IM receiver 202 includes a speed sensor 203 to be described furtherbelow. The OOSR router also includes centralized Network ProcessingModule(s) (NPMs) responsible for the processing ofnon-time-sensitive/exception condition tasks, e.g., routing maintenance,management, error protocol operations and a QoS preserving opticalNetwork Interconnect Module (NIM) 208, comprising of the IMs and theNPMs interconnected together as nodes in a k-ary n-cube viabi-directional optical links. The NIM is provided having an appropriatearchitecture and/or arbitration, scheduling or routing algorithms, forpreservation of QoS. It should be appreciated that in the case where theNIM is provided as a BMIN, the BMIN can be provided from opticalchannels and optoelectric hardware-based switching elements.

The OOSR architecture is similar to the OSR 164 of FIG. 8 in thatnetworks (including but not limited to IP networks) using OOSR routersalso rely on the use of general-purpose IM receivers and has an RSNA asthe additional (separate) network entity for processing the QoS ControlPlane and some data plane functions for “special needs” packet headers.The RSNA dynamically provides resource and reservation informationpertaining to overall network view, rather than in a hop-by-hop manner.An “edge” router's RSNA connection must always be on and a“hub/backbone” router's RSNA connection may be off. One differencebetween the OOSR router and the OSR router 164, however, lies in thechip and interchip interconnection technology. In particular, the OOSRutilizes an optoelectronic chip with optical interchip interconnect inboth the RSNA and the IM as further explained below.

Similar to the RSNA for the OSR architecture, the internal (packetheader processing) functions of the RSNA in the OOSR architecture areindependent hardware-based functions. Each of these functions has itsown dedicated data structure and storage capacity needed to perform itsdedicated operation and is implemented using (re-)programmable hardwarecomponents such as FPGAs. However, instead of electrical interconnectionamongst the logic level (FPGA) chips, these chips are opticallyinterconnected to one another in pipeline fashion where functionalinter-dependence exists i.e., optical (interchip) interconnect withinthe RSNA (see FIG. 10). The RSNA continues to provide a scalableseparate QoS/CoS control plane server and information base along withsome data plane functionalities.

As FPGAs are electrical components, optoelectronic FPGAs are used tointerface with the optical interchip interconnect, rather than theelectronic FPGA chips. Hence, packet headers processing are done usingoptical interconnect amongst the optoelectronic FPGA chips as ittraverses through the RSNA.

On the other hand, the IM (as explained about the two paths 230 and 232in conjunction with FIG. 10 above) is implemented with two types of chiptechnologies and two types of interchip interconnect technologies. Thechip technologies are respectively, optoelectronic FPGA and pureelectronic FPGA; whilst the interchip interconnect technologies arerespectively all-optical and all-electrical Hence the IM interior iscomposed of two distinct parts. An optical part with optoelectronic(FPGA) chips optically interconnected to one another and anall-electrical part with regular electronic (FPGA) chips electricallyinterconnected to one another. Consequently, the IM's 201 includeoptical to electrical O-E conversion 205 at entry to the IM andelectrical to optical converters (E/O) 206 at exit from the IM (into theoptical SF 208). Thus, the IM utilizes dual hardware technologies(optical and electrical). The purpose of these dual (chip andinterchip-interconnect) hardware technologies will become apparent fromthe description below. Similar to the OSR using k-ary n-cube SF, theinterconnection technology of the NIM 208 remains all-optical as alludedto above.

It should be appreciate that even though the E-O and O-E conversions aredrawn as part of the IM receiver in FIG. 11 (and not as part of the IM),in some embodiments it may be desirable or necessary to implement asystem either way. That is, the E-O and O-E conversions can beimplemented as either part of the IM or as part of the IM receiver.

The IM receiver 202 includes the speed sensor 203 that can detect therelative speed of the IM connection. In one embodiment, for purposes ofefficient network hardware expansion/upgrade, (i.e. an upgrade which canbe accomplished via re-programming a chip) the speed sensor isimplemented as a speed sensing algorithm in a programmablehardware-based component. It should be appreciated, that thehardware-based sensor algorithm applies only to the IM receiver withinthe router and not the RSNA in this OOSR architecture, which will alwaysroute traffic using the optical interconnect and hence no need for speedsensing. This is due to the fact that incoming traffic to the RSNA willalways occur at high speed, as an RSNA serves >1 router within itsnetwork domain of responsibility for a network using the OOSR routerarchitecture.

Based upon sensor result, the IM receiver 202 will perform a “routingspeed adjustment” (i.e. the receiver can “speed up” or “slow down”). Itwill cause the packets to be routed via electronic inter (FPGA) chipinterconnect through the IM using the optical-to-electrical (O-E)conversion 205 at the link interface if the detected speed is below acertain “watermark” (e.g., at or below OC 12), i.e., an electricalinterchip interconnect through the IM 201 will be provided.

If, on the other hand, the detected incoming packet speed is above the“watermark”, the IM receiver 202 will cause incoming packets to berouted via optical interchip interconnect 210 through the IM. It shouldbe noted that whichever (physical) hardware path the packet is routedthrough, packets in each hardware path will still have the three logicalprocessing paths time-sensitive, non-time-sensitive or RSNA as describedabove to process through depending upon the packet processingrequirement as indicated by information in the header.

With the aid of these speed-sensing general-purpose IM receivers 202 toperform routing speed adjustment through the IM according to incomingpacket speed and a separate RSNA function with optical interchipinterconnect, the interior architecture is uniform across all OOSRrouters, regardless of its placement in the network hierarchy. Hence,the property of “bi-directional reconfigurability” is retained and isusable either during emergency situations or network hardwareexpansion/upgrade.

It should be noted that all functional descriptions for the OSR router164 (FIG. 8) with respect to IMs, NIM, NPM and RSNA apply to the OOSRrouter. Two differences are that for the optical interconnect part ofthe IM in the OOSR router, no E/O device is needed at the optical SFinterface within the IM (however, as in OSR, E/O device such as VCSELsis needed for the electrical part of the IM to interface to the opticalchannel interconnect). Similarly, since the packet header processingfunctions in the OOSR RSNA 220 (FIG. 10) are also implemented usingoptoelectronic (FPGA) chips with optical interchip interconnect 222(FIG. 10), no E/O devices are needed for these functions to interface tothe all-optical interconnection channels within the RSNA.

Use of the intelligent IMs provides the OOSR router having a distributedforwarding functionality. Also the OOSR router architecture providessegregation of time-sensitive from non-time-sensitive tasks amongstRSNA, IM and Network Processing Module.

An OOSR router architecture serves dual purposes. First, it provides atool for router reconfigurability. A router serving at the “edge” willlikely experience lower traffic rate (e.g., currently ≦OC48) than thatexperienced in a “hub” or “backbone” router. Hence a router placed inthe “edge” layer makes use of the electrical interconnect through theIMs; whereas if that same router is placed in the “hub” or “backbone”layer, would make use of the optical interconnect through the IM.Second, it serves as “transition” architecture to usher in technologicalevolution from electronic devices, with its ultimate limitation inelectronic processing speed, to optical devices

The OOSR is appropriate in applications in which there exists a customerbase at the “edge” of an ISP network with aggregate speed sufficientlylow such that the service provisioning for these customers could stillbe economically accommodated by electronic routing through the routerIM. Moreover, the additional intra-IM optical interconnect capabilitynot only allows efficient transition to optical technology, butoperating with the intra-IM electronic interconnect counterpart alsoallows an OOSR router to scale up to much higher magnitude in throughputthan electronic intra-M interconnect alone. The latter advantage isagain due to the notion that there exists two different paths each witha different hardware technology through the OOSR router, making itpossible for the router to process traffic with input speed of very lowto very high magnitude. One “lower” speed hardware path composes ofelectronic chips and electrical interchip interconnect through the IM,and optical channel interconnect amongst the IMs through the router. Theother “higher” speed hardware path composes of optoelectronic chips withoptical interchip interconnect through the IM, and optical channelinterconnect amongst the IMs through the router.

Finally, a potential advantage of the OOSR architecture, is that theproblem of preserving constant packet delivery delay variation (such asprevention of jitteriness in voice packets) through the SF could beovercome, with optical interchip interconnect within the IM.

All references cited herein are hereby incorporated herein by referencein their entirety.

Having described preferred embodiments of the invention, it will nowbecome apparent to one of ordinary skill in the art that otherembodiments incorporating their concepts may be used. It is felttherefore that these embodiments should not be limited to disclosedembodiments, but rather should be limited only by the spirit and scopeof the appended claims.

1. A method of re-configuring a router comprising: determiningthroughput requirements of a hub/backbone router; grouping together aplurality of edge routers to meet the throughput requirements of thehub/backbone router; turning on a hub/backbone mode of at least one ofthe plurality of edge routers; and adjusting I/O line rates of each ofthe at least one of the plurality of edge routers such that the linerates in each of the at least one of the plurality of edge routers aresuitable for operation of the router in hub/backbone mode.
 2. The methodof claim 1 wherein turning on a hub/backbone mode of at least one of theplurality of edge routers includes one of: (1) turning off a connectionbetween the router and an RSNA; and (2) leaving on a connection betweenthe router and an RSNA and providing a passive QoS/CoS provisioning forpackets entering the router.
 3. The method of claim 2 wherein leaving ona connection between the router and an RSNA comprises: turning on ahub/backbone mode of the router; automatically sending a message toinform an RSNA that the router is in hub/backbone mode; listening tocommands from the RSNA; and executing commands from the RSNA.
 4. Themethod of claim 2 further comprising: turning off the hub/backbone modeof the router; turning on the edge mode of the router such that the RSNAprovides active CoS/QoS provisioning for packets entering the router;and adjusting I/O line rates of the router such that the I/O line ratesare suitable for operating the router as an edge router.
 5. The methodof claim 1 wherein adjusting I/O line rates of at least one of theplurality of edge routers includes adjusting I/O line rates of at leastone of the plurality of edge routers by reprogramming an IM receiver ineach of the at least one of the plurality of edge routers such that theI/O line rates are suitable for operating the router as a hub/backbonerouter.
 6. A method of reconfiguring a router, the method comprising:turning on an edge mode of a hub/backbone router; and adjusting I/O linerates of the hub/backbone router such that the I/O line rates aresuitable for operating the hub/backbone router as an edge router.
 7. Themethod of claim 6 wherein turning on the edge mode of the hub/backbonerouter comprises turning on a connection though which the routercommunicates with an RSNA such that the RSNA provides active CoS/QoSprovisioning for packets entering the router.
 8. The method of claim 7wherein the router communicates with an RSNA via a request/responsesession.
 9. The method of claim 7 further comprising: turning off theedge mode of the router; turning on the hub/backbone mode of the router;and adjusting I/O line rates of the router such that the I/O line ratesare suitable for operating the router as a hub/backbone router.
 10. Themethod of claim 6 wherein adjusting I/O line rates of the hub/backbonerouter includes adjusting I/O line rates of the hub/backbone router byreprogramming an IM receiver in the hub/backbone router.
 11. The methodof claim 10 wherein reprogramming the IM receiver in the hub/backbonerouter to adjust I/O line rates of the hub/backbone router includesreprogramming the IM receiver to provide a desired emergencyfix/response turn-around time.
 12. A method comprising: providing aconfigurable router which is configurable both as an edge router and asa hub/backbone router; providing a router special needs agent (RSNA),turning on active CoS/QoS provisioning between the configurable routerand the RSNA such that the RSNA provides active CoS/QoS provisioning forpackets entering the configurable router and such that the configurablerouter operates in an edge router mode; and turning off active CoS/QoSprovisioning by the RSNA such that the configurable router operates in ahub/backbone router mode.
 13. In a network which includes at least oneedge router, a router special needs agent (RSNA) in communication withthe edge router and at least one of: (1) a hub router in communicationwith the edge router and said RSNA; and (2) a backbone router incommunication with the hub router and the RSNA, a method ofreconfiguring comprising: configuring a selected one of the at least oneedge routers to function as a selected one of: (1) a hub router; and (2)a backbone router, said configuring including adding at least oneadditional interface module (IM) and one or more associated channels inthe selected edge router; and turning off active CoS/QoS provisioning bythe RSNA such that the selected edge router operates as the selected oneof the hub router and the backbone router.
 14. The method of claim 13further comprising reconfiguring the selected one of the at least oneedge routers, said reconfiguring including enabling active CoS/QoSprovisioning between the selected one of the at least one edge routersand the RSNA such that the selected one of the at least one edge routersno longer functions as one of a hub and a backbone router.
 15. In anetwork including at least one edge router, a router special needs agent(RSNA) in communication with said edge router, and at least one of: (1)a hub router in communication with the edge router and the RSNA; and (2)a backbone router in communication with the hub router and the RSNA, amethod of reconfiguring comprising: configuring a selected one of thehub router and the backbone router to function as an edge router, saidconfiguring including enabling active QoS/CoS provisioning between theselected one of the hub router and the backbone router and the RSNA. 16.The method of claim 15 further comprising reconfiguring the selected oneof the hub router and the backbone router by turning off active CoS/QoSprovisioning by the RSNA between the selected one of the hub router andthe backbone router and the RSNA such that the selected one of the hubrouter and the backbone router no longer functions as an edge router.